A 640×512 CMOS Image Sensor with Ultra Wide Dynamic Range Floating-Point Pixel-Level ADC

A 640×512 CMOS image sensor with an 8b bit-serial Nyquist rate ADC per 4 pixels achieves 10.5 × 10.5 μm pixel size at 29% fill factor in 0.35 μm CMOS technology. Binary floating point output with measured dynamic range of of 65536 : 1 is achieved by multiple sampling at exponentially increasing exposure times. D. Yang, A. El Gamal, B. Fowler, and H. Tian 1 Dynamic range, defined as the ratio of the largest nonsaturating signal to the standard deviation of the noise under dark conditions, is a critical figure of merit for image sensors. The dynamic range of an image sensor is often not wide enough to capture scenes with both high lights and dark shadows. This is especially the case for CMOS sensors, which, in general, have lower dynamic range than CCDs. Several approaches have been proposed to enhance the dynamic range of a CMOS APS. In [1] dynamic range is enhanced by increasing well capacity one or more times during exposure time. Another approach, which achieves consistently higher SNR, is multiple sampling. Here the scene is imaged several times at different exposure times and the data is combined to construct a high dynamic range image. For this approach to work at reasonable capture times, readout must be performed at speeds much higher than normal APS speeds. In [2] an APS with two column parallel signal chains is presented. The sensor can simultaneously read out two images, one after a short exposure time T and the other after a much longer exposure time, e.g. 32T . Two images, however, may not be sufficient to represent the areas of the scene that are too dark to be captured in the first image and too bright to be captured in the second. It is difficult to extend the scheme to simultaneously capture more than two images, since more column parallel signal chains must be added at considerable area penalty. In this paper we demonstrate, using a 640×512 image sensor with Nyquist rate pixel level ADC implemented in a 0.35μm CMOS technology, how pixel level ADC enables a highly flexible and efficient implementation of multiple sampling. Since pixel values are available to the ADCs at all times, the number and timing of the samples as well as the number of bits obtained from each sample can be freely selected without the long readout time of APS. Typically, hundreds of nanoseconds of settling time per row are required for APS readout. In contrast, using pixel level ADC, digital data is read out at fast SRAM speeds. This demonstrates yet another fundamental advantage of pixel level ADC — the ability to programmably widen dynamic range with no loss in SNR. The 640 × 512 sensor employs the MCBS ADC technique described in [3]. Each 2 × 2 block of pixels share a 1-bit comparator/latch pair. The signals required to operate the ADCs are globally generated by off chip DAC and digital control circuitry. The ADC is bit D. Yang, A. El Gamal, B. Fowler, and H. Tian 2 serial and each bit is generated by peforming a set of comparisons between the pixel values and a RAMP signal. The bits are generated independently and in any order, and the data is read out as bit planes [3]. We consider the implementation of multiple sampling for exponentially increasing exposure times T , 2T , 4T, . . . 2T . Each sample is digitized to m bits. The digitized samples for each pixel are combined into an m+ k bit binary number with floating point resolution. The number can be converted to a floating point number with exponent ranging from 0 to k and an m bit mantissa in the usual way. This increases the sensor dynamic range by a factor of 2, while providing m bits of resolution for each exponent range of illumination. An important advantage of this scheme, over other dynamic range enhancement schemes [1], is that the combined digital output is linear in illumination. This not only provides more information about the scene, but also makes it possible to perform color processing in the standard way. To illustrate how our sensor implements this scheme, consider an example with k = 2 and m = 2. Figure 1 plots the output pixel voltage versus time for three constant illuminations I1, I2, and I3, assuming linear photon to voltage response, and saturation voltage Vmax. The voltage is sampled after exposure times T , 2T , and 4T . Each sample is digitized to 2 bits using binary code as shown in the figure. The first sample is digitized to x1x2, e.g. 11 for I1. The second is also digitized to 2 bits. Since the second sample is twice the value of the first, the 2 bits are x2x3 if the sample is not saturated, i.e. < Vmax, and 11 if it is. In either case, the ADC only needs to generate the least significant bit x3. Similarly, for the third sample at 4T only the least significant bit x4 needs to be generated. Table 1 lists the bit values for each illumination and the corresponding binary floating point representation. This scheme can be easily extended to any exponent k and mantissa m. The first sample is digitized to m bits, then only the least significant bit is generated from each consecutive sample. Thus as long as the sensor photon to voltage response is linear only m + k bits need to be read out, which is the minimum required. This further reduces readout time. Often more bits need to be read out from the samples to correct for sensor nonlinearity, offset, and noise. A schematic of four pixels sharing a 1-bit comparator/latch pair, and the column sense amplifier is shown in Figure 2(a). The comparator/latch circuit is described in [3]. An antiD. Yang, A. El Gamal, B. Fowler, and H. Tian 3 blooming transistor is connected to each photodetector to avoid blooming during multiple sampling. The sense amp is designed for high speed, low noise and power. A charge amplifier is used to minimize bitline voltage swing. To sense the bitline, capacitor Mc is first reset, and the current from the pixel and transistor M7 is integrated. The sensed bit is latched using a flip flop. The tristate inverter is part of a 10:1 mux. A scene is imaged one quarter frame at a time as illustrated in Figure 2(b). Before capturing a quarter frame the sensor is reset. Sample and hold is used to ensure that the signal does not change during ADC. Each quarter sample image is read out one bit plane at a time [3]. The quarter images are, then, combined to form a quarter frame, and the quarter frames merged into a high dynamic range frame. A photomicrograph of the sensor is provided in Figure 6. Table 2 summarizes the main sensor chip characteristics. Figure 3 plots the pixel output voltage vs. time under constant illumination. Note that it is quite linear. A scene with measured dynamic range ≥ 10 is illustrated in Figures 4 and 5. The scene was sampled 9 times, at T = 3.3ms, 2T, . . ., and 256T , and a 16-bit image was reconstructed. Since we cannot print gray scale images of more than 8 bits, in Figure 4 we display four 8-bit slices of the image. The first is a plot of the 8 most significant bits of the 16-bit image, where only the roof and the front of the dollhouse is seen. The 2nd was obtained by brightening the image by 8x. Several objects inside the house are revealed. These objects become more visible in the 3rd image, after the original image was brightened by 32x. The 4th image was obtained by brightening the image by 256x. A table and a man hiding in the dark area next to the house appear. Figure 5 is an 8-bit plot of the log of the 16-bit image. Acknowledgments: The authors acknowledge the support of HP, Intel, Kodak, Canon, and Interval Research and the assistance of X. Liu, and M. Godfrey. D. Yang, A. El Gamal, B. Fowler, and H. Tian 4